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Sufficient conditions are developed for a class of generalized Polya urn schemes ensuring exchangeability. 
The extended class includes the Blackwell-MacQueen Polya urn and the urn schemes for the two-parameter 
Poisson-Dirichlet process and finite dimensional Dirichlet priors among others. 



1. INTRODUCTION 

By making use of a remarkably simple generalized Polya 
urn scheme, Blackwell and MacQueen ([1]) described an el- 
egant alternate way to approach the Ferguson Dirichlet pro- 
cess (101). Let Xi,X2, ... be a sequence of random elements 
on a complete separable metric space X defined by 

¥{X X e •} = MOM*) (D 

and 

P{I i+1 6-|I ll ...,I i } = tt Q// li (4 »>1, (2) 

where fii(-) — /i(-) + Y?j=i ^x t (') an d A* is a finite non-null 
measure on X. Blackwell and MacQueen called such a se- 
quence a Polya sequence with parameter \x. They showed 
that if X\, X2, ■ ■ ■ was such a sequence, then: 

(a) / Hi(X) converges almost surely to a discrete ran- 
dom probability measure p* . 

(b) p* is the Ferguson Dirichlet process with parameter p. 

(c) Given p,*, X x , X 2 , ■ ■ . are independent with distribution 

Result (c) shows that the Blackwell-MacQueen Polya se- 
quence is exchangeable, while (b) shows that the sequence is 
an infinite sample from the Dirichlet process. Thus (a) and 
(b) combined show that the Polya urn defined by Q and (0 
is a way to draw values from the Dirichlet process. More- 
over, (a) shows that the Dirichlet process is the limit for the 
urn distribution pi(-)/ l^i(X), thus providing an alternate way 
to characterize the Dirichlet process. These facts are far more 
difficult to prove than the contrapositive result which starts 
with a sample from the Dirichlet process and shows that such 
a sample must be exchangeable and can be constructed from 
a Polya urn. The latter result follows from elementary prop- 
erties of the Dirichlet process which we now describe. Let 
Xi , X2 , ... be a sequence derived from a Dirichlet process 
with parameter p, i.e: 

(Xi\P) P, i = 1,2,... 
P ~ p*. 
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It was shown in |2] that p*(-\X x , . . . , Xi), the posterior for p* 
based on the first i observations Xi , . . . , Xi, is also a Dirichlet 
process, but with an updated parameter pi (see @], Section 
3.2, for a proof using a Laplace functional argument). An 
immediate consequence of this is that 

P{X i+1 G -\Xi,...,Xi} 

= I ¥{x l+1 e ■\x 1 ,...,x l ,p}p*(dP\x 1 ,...,x i ) 

= J P(-)p*(dP\X 1 ,...,X i ) 

= m»(OM(*)) 

and, thus, X\ , X2 , ■ ■ ■ can be defined by the Polya urn de- 
scribed by Q and (0 (the fact that the sequence is exchange- 
able follows by definition). 

Put another way, elementary properties for the Dirich- 
let process shows us that the prediction rule, that is, the 
conditional distribution for Xi +X given Xi,. . . ,Xi, corre- 
sponds to an exchangeable generalized Polya urn distribution 
Pi(-)/ pi(X). This type of direct result is somewhat unique as 
it is generally hard to derive simple explicit prediction rules 
for a general random discrete probability measure. Instead, 
another way to approach the problem is in the direction stud- 
ied by Blackwell and MacQueen. Thus, it is natural to wonder 
what types of Polya urn schemes other than Q and (|2ji lead 
to: (i) an exchangeable sequence Xi, X2, ■ ■ ■ and (ii) an urn 
distribution with a limiting random discrete probability mea- 
sure? Sufficient and necessary conditions for (i) have been 
given in |3[ (cf Theorem 2) in terms of what is called the 
exchangeable partition probability function (EPPF), a sym- 
metric non-negative function which characterizes the distri- 
bution of an exchangeable partition on the positive integers 
{1,2,...}. In this paper, however, we take a more direct ap- 
proach to answering (i) (and consequently (ii)), by introduc- 
ing an Exchangeability Condition (Section 3) which puts con- 
straints on the manner in which the urn scheme selects new 
values or chooses a previously sampled value. While these 
conditions will be shown only to be sufficient to ensure (i), 
they have the advantage that they are simpler to understand 
then conditions stated in terms of the EPPF. The proof should 
also be readily accessible to non-experts to this area. Our Ex- 
changeability Condition is shown to hold for several important 
generalized Polya urns, including those for the two-parameter 
Poisson-Dirichlet process (flljn as well as the class of finite 
dimensional Dirichlet priors (|5]). Corollary 1 of Section 3, 
our main result, summarizes our results. 



2. NOTATION AND BACKGROUND 
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2.1. Non-exchangeability over discrete spaces 



We begin by introducing some notation necessary to ex- 
plain our generalized Polya urn schemes. Let = {Cj^ : 
j = 1, . . . , n(pi)} denote a partition of {1, . . . , i} where 
Cj.i is the jth set of the partition. Write e^j for the car- 
dinality of Cj_i. Thus, Pi is a partition made of n(pj) 

sets and Yl'j^i e j,i = *■ Let X*,X 2 , ■ ■ ■ denote the se- 
quence of unique values in the order of their appearance from 
Xx,X%, . . . and let p; be a partition of {1, • • • , i} recording 
the clustering of the first i observations X\, . . . , Xi. By this 
we mean Xi — X* for each I e Cjj, where j = 1, . . . , n(p*). 

Let v denote a non-null probability measure over X. We 
will consider sequences X\, X 2) . . . defined by 

F{X 1 G •} = KO (3) 

and 

F{X i+1 e-\X u ...,Xi} (4) 
"(Pi) 

= — — "(0 + E — r\ — <M0> * > i, 

where 

: = qj,i( e i.i, e„( Pi ).i), 9o,i := ?o,i(ei,i, ■ • ■ , e n ( pt )^) 

(5) 

are non-negative real valued symmetric functions depending 
only upon {e M , . . . ,e„ (Pi) l }. 

The form for qo j and q^i in (|5} is suggested by Theorem 1 
of [3] which states that for X±, X 2 , ■ • ■ to be exchangeable, 
the functions q^ : i an d must be almost surely equal to some 
function of the partition p^; or equivalently, they must be some 
function of the cardinalities e^. For example, observe that 
the Blackwell-MacQueen Polya sequence (with parameter fi) 
corresponds to the choices qo j = (J,(X), qj = ej j and v(-) = 

MOM*). 

In proving our general result, an important technical con- 
dition that we will need to address concerns the choice for v. 
We say that v is non-atomic if v{x} = for each i£l One 
of the unique features of the Blackwell-MacQueen Polya urn 
scheme is that it yields an exchangeable sequence regardless 
of whether v is non-atomic. For example, \f X — {1, . . . , r} 
is a finite sample space and /i is a finite discrete measure over 
X, then and (|2j implies that X\ , . . . , Xi is the result of 
successive draws from an urn originally having p,(l) balls of 
color I, and following each draw for a ball, the ball is replaced 
and another one of its same color is added to the urn. It fol- 
lows that 

FIX, =x 1 ,...,X i = x i }= — ^ f[ M (0 [n(0] , 

where n{l) denotes the number of x's equal to I and afi — 
a(a + 1) • • • (a + i — 1) (note: = 1). Observe that since 
the right-hand side is a symmetric function of (xi , . . . , Xj), it 
follows automatically that Xi, . . . , Xi is exchangeable. This 
fact was not lost on Blackwell and MacQueen. Indeed, the 
key to their proof relies on the fact that their Polya urn scheme 
produces an exchangeable sequence for finite sample spaces. 



However, prediction rules for random discrete measures of- 
ten break down when X is allowed to be a finite sample space. 
A good example is the two-parameter Poisson-Dirichlet pro- 
cess discussed in 1 1 1]. This is the random discrete probability 
measure whose prediction rule for a non-atomic v corresponds 
to the choices 

Qo,i = 9 + cm(pi) and qj ti = e^j - a, 

where < a < 1 and 9 > -a. See (H| and also @1, 
II (II for further details. Setting a = and 9 = fi(X) leads 
to the Blackwell-MacQueen Polya sequence with parameter 
/i = 9v, and as discussed produces an exchangeable sequence 
without constraint to v. In general, however, if a ^ 0, ex- 
changeability breaks down if X is allowed to be a finite sam- 
ple space and v is atomic. This can be easily demonstrated 
by the following counter-example. Let X = { 1 , . . . , r} where 
r > 2 and suppose that v{l) = 1/r for each I = 1, . . . ,r. 
Then, 

F{X 1 = 1,X 2 = 2,X 3 = 1} 
= F{X! = 1} x F{X 2 = 2\X X } 

xF{X 3 = 1,X 3 = Xx,X 3 ± X 1 \X 1 ,X 2 } 

(d + a)((9 + 2a)/r+l-a) 

= r 2 (9 + l)(9 + 2) ' 

Note that the last expression in the middle equation underlies 
the problem with working with an atomic measure. Here the 
conditional event that X 3 = 1 occurs if we choose the pre- 
vious value Xi or if we choose the value X = 1 randomly 
from v. This wouldn't be a problem with a non-atomic proba- 
bility measure since the probability of obtaining a previously 
observed Xi value is zero under v. But this leads to a break- 
down of exchangeability for an atomic measure. Consider the 
probability, 

F{X 1 = l,X 2 = l,X 3 = 2} 
= F{X 1 = l}xF{X 2 = l,X 2 = X 1 ,X 2 ^X 1 \X 1 } 
xF{X 3 = 2\X 1 ,X 2 } 

({6 + a)/r + l-a) {9 + a) 
= r 2 (9 + l)(9 + 2) ' 

Thus, P{Xi = 1,X 2 = 2,X 3 = 1} ^ P{Xi = 1,X 2 = 
1, X 3 = 2} unless a = 0. This shows that only the Blackwell- 
MacQueen Polya urn is exchangeable in general for the two- 
parameter process. 

3. MAIN RESULTS 

Thus, given the technical difficulties in working with 
atomic measures, we will hereafter restrict attention to non- 
atomic measures v. Our results will also rely on the following 
key conditions for the functions go,« an d Qj,i appearing in Q 
and 0J. 
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Exchangeability Condition. For each i > 1, qj :i = ^p(e^i) 
and qo t i = f/'o( n (Pi))> where %p and ipo ore some fixed non- 
negative real valued functions. Furthermore, for each parti- 
tion pi of{l, ...,«} 

"(Pi) 

E «7.< = £(0 > ( 6 ) 
3=0 

where £ is some fixed positive real valued function. 

These conditions are satisfied by many interesting gener- 
alized Polya urn schemes, which we now list. By satisfy- 
ing the Exchangeability Condition, Theorem 1 (stated later) 
shows that each of these urns (subject to a non-atomic v) are 
exchangeable. 

1. Independent and identically distributed sampling. This 
is of course the most obvious form of exchangeability 
and follows with choices <7o,i = 1 and qj t i = 0. 

2. N values selected at random. Let N > 1 be a posi- 
tive integer and let go,i = (N ~ n(pi))I{n(pi) < N} 
and qj_i = 1. Observe that <7o,i becomes zero when 
n (Pi) > N, which restricts the urn sequence from hav- 
ing more than TV distinct values. Note that condition 



for the finite dimensional Dirichlet prior Vn defined by 



is satisfied because Ej=o Qj 



N. 



3. The Blackwell-MacQueen Polya sequence with param- 
eter /i. This corresponds to qj.i = e^u ?o,i — and 

Z;L p o ) q^ = ii(x) + i . 

4. The two-parameter Poisson-Dirichlet process. As dis- 
cussed, this corresponds to choices go,; = & + cm(p;) 
and qjj — ej i — a, where < a < 1 and 9 > —a. 

Thus, X^=o = Q + i- This is the prediction rule 
for the random discrete probability measure V defined 
by 

V(-) = V 1 6 Zl (-) (7) 

oo 

+ W - V 2 ) • ■ • (1 - V k ] 5 Zh {-), 

k=2 

where {14} are i.i.d Beta(l — a, 6 + ka) random vari- 
ables, independent of {Z k }, which are i.i.d with law v. 
See fllll for details. Observe that by setting a = we 
end up with the Dirichlet process with parameter p, = 
Ov. In this case, Q corresponds to the stick-breaking 
representation for the Dirichlet process. See |4[ for 
background on stick-breaking priors. 

5. Finite dimensional Dirichlet priors (Fisher's model). 
Let N > 1 be a positive integer and let go,i = 0(1 — 
n(pi) /N) 1 {nip,) < N} and q jti = e jti + 9/N, where 

6 > 0. Then Qj,* = 9+i which sati snes ©. Ob- 

serve that the choice for #o,i restricts the process from 
having more than N distinct values. One can show that 
the values qo.i and qj^ correspond to the prediction rule 



N 



k=l U k 

where {Gk} are i.i.d Gamma(0/iV) random variables, 
independent of {Zk}, which are i.i.d with law v. 
See |3, O and jj for further details. Also see H 
who showed that Vn is a weak limit approximation to 
the Dirichlet process. 



3.1. Exchangeability 

We now show that our Exchangeability Condition is suf- 
ficient to ensure that the sequence defined by and @ is 
exchangeable. 

Theorem 1. If v is a non-atomic (and non-null) probabil- 
ity measure over X and the Exchangeability Condition holds, 
then Xi , X 2 , ... is exchangeable. 

Proof. Let i > 1 (the case i = 1 is obvious) and let 
dx\,...,dXi denote a sequence of differentials, some of 
which can be equal. Let p; = {Cj.i : j — l,...,n(pj)} 
be the partition of {1, . . . , i} which records the clustering of 



dxi, . . . , dx{. That is, if dx\, 



, dx 



"(P.) 



denote the unique 



values of dx\, . . . , dxt, then dxi — dx* for each I e Cjj. 
As before, write e^j for the cardinality of Cj %. For notational 
convenience set -0(0) = 1. 

It follows from the assumption that v is non-atomic, and 
upon using and 0, that 



P{Xi 6 dxi,...,Xi G dxi} 
*{Xj edXj\Xi,... 



(8) 



n 



Xj-i} 



n(pi)-l n(p<) 



D^ 1 n Mk) J] { v{dx^)I{dxi = dx* : I S Cj, t } 
k=i j=i ^ 



x ip(ej 



where the first product (in square brackets) follows from the 
assumption that go,j = i J o( n (Pi)) (note: if n(pj) = 1 the 
product is assumed to be 1), while the second product uses the 
assumption that qj^ — tjj(ej^). The expression Di appearing 
in (|SJ| is a normalizing constant. By (jSJl, it can be seen that 
Di — £(1) x •• • x £(i — 1). Thus, deduce that the right- 
hand side of (|8j is a symmetric function of [dx\,... ,dxi), 
and hence that X\, . . . , X, is exchangeable. □ 

Remark 1. As a special case, the expression (0 yields the 
well known joint density for the Dirichlet process: 



^{Xi e dxi, . . . ,Xi 6 dxi} = 



n(p0 



x Yl ( v(dx*)I{dxi = dx* : I e C jti ) (e jti - 1)' 

3=1 ' 
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(Substitute £(j) = fi(X) + j, ip(j) = j and i/) (j) = fi(X) 
into ©J. 

3.2. The Blackwell-MacQueen generalization 

By appealing to Proposition 1 1 of ifToll . in combination with 
Theorem 1, we obtain the following corollary which is a gen- 
eralization of the Blackwell and MacQueen result. 

Corollary 2. Let X± , X2 , ... be the sequence defined by (0 
and @ where v is a non-atomic (non-null) probability mea- 
sure over X and {go,ij Qj,i} satisfy the Exchangeability Con- 
dition. 

(a) Let -Fj+i denote the conditional distribution for Xj+i 
defined by (0. Then, i^+x — * V* in l\-distance, where 
V* is the random probability measure defined by 

^0 = £p^*;(0 + (i-£p>(-)> 

3 j 

where pj — linij—xx, ej,%/i. 

(b) {X*} are Ltd v and independent of {pj}. 



(c) Given V* , X\,Xi^ . . . are independent with distribu- 
tion V*. 

(d) Ifqo,i/£(i) — * 0, then V* is discrete with probability 
one; i.e. V*{-) = J2 3 Pj %(•)• 

Proof. Theorem 1 ensures that Xi,X2, ... is exchangeable. 
Thus, (a), (b) and (c) follows from de Finetti's representation 
for exchangeable sequences. See Theorem 6 of |9] and Propo- 
sition 1 1 of lHoll . To prove (d) we use a theorem of 0] which 
states that if X\, X2, ... is an exchangeable sequence from a 
random measure V* , then V* is discrete with probability one 
if 

a, = ¥{X, +1 is different than X x , . . . , JQ} ^ 0. 

See H2I1 . Section 1.6, for a proof. Thus, (d) is proven since 

a-t = <7(m/£W- □ 

Remark 2. A little bit of work shows that each of our ex- 
amples listed earlier (excluding our first example for the i.i.d 
case) are examples of generalized Poly a urn schemes which 
satisfy condition (d). Thus, each produce exchangeable se- 
quences from a random discrete probability measure. 
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